Graph Coloring in Optimization Revisited

Algorithms for solving numerical optimization problems and nonlinear systems of equations that rely on derivative information require the repeated estimation of Jacobian and Hessian matrices. The problem of minimizing the number of function evaluations needed to estimate the derivative matrices, within a finite difference method or an automatic differentiation method, has been formulated as a collection of (specialized) graph coloring problems. These formulations have led to the successful use of graph coloring algorithms in optimization software. We survey the historical development of graph coloring formulations and algorithms in the context of estimating derivatives in optimization. (We believe this is the first such survey.) A dozen variations of derivative matrix estimation problems exist, depending on the matrix to be evaluated, whether all or only a subset of the nonzero matrix elements need to be evaluated, and on the details of the estimation techniques. The plurality of problems has led to a plethora of techniques, and the inherent similarity of the various estimation problems has thus far been obscured. This fragmentation also makes it difficult to identify a generic formulation, thereby hindering the development of algorithms and flexible software. Given the incremental nature of advances in research, this fragmentation was perhaps inevitable. With the advantages of about twenty years of hindsight, we describe a unified graph theoretic framework for solving the various matrix estimation problems, based on a graph-theoretic characterization Computer Science Department, Old Dominion University, Norfolk, VA 23529-0162 USA. {assefaw, pothen}@cs.odu.edu Department of Informatics, University of Bergen, N-5020 Bergen, Norway. [email protected]